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Abstract. - We study the robustness of functionals of probability distributions such as the Renyi 
and nonadditive S q entropies, as well as the g-expectation values under small variations of the 
distributions. We focus on three important types of distribution functions, namely (i) continuous 
bounded (ii) discrete with finite number of states, and (iii) discrete with infinite number of states. 
The physical concept of robustness is contrasted with the mathematically stronger condition of 
stability and Lesche-stability for functionals. We explicitly demonstrate that, in the case of 
continuous distributions, once unbounded distributions and those leading to negative entropy 
are excluded, both Renyi and nonadditive S q entropies as well as the g-expectation values are 
robust. For the discrete finite case, the Renyi and nonadditive S q entropies and the q-expectation 
values are robust. For the infinite discrete case, where both Renyi entropy and g-expectations 
are known to violate Lesche-stability and stability respectively, we show that one can nevertheless 
state conditions which guarantee physical robustness. 
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Introduction. — Thermostatistical quantities such as 
entropy are expressed as functionals of probability distri- 
butions. For these quantities to be physically meaningful 
they should not change drastically if the underlying distri- 
bution functions are slightly changed. In practical terms, 
the unavoidable experimental uncertainty in determining 
the distribution function should not cause the thermody- 
namical quantities to fluctuate wildly, or even diverge. It 
is therefore of elementary interest to clarify and check that 
thermodynamical quantities are robust under small varia- 
tions of the distribution functions. We define probability 
distributions on a set of W discrete states, p — {pi}]^. 
Let us denote a variation by p\ = pi+ Spi, the L\ distance 
being ||p -p' ||i = J2Zi \Pi "Kl- 
in this context, almost three decades ago, Lesche has 
introduced a definition of stability of functionals [1] . There 
a functional Q[p] (e.g., an entropic form) is called stable 
[Lesche- stable, as often referred to nowadays) if, for every 
e, one can find a 8 such that for all W and for all p and p' 
one has 



Ip-p'Hi < 6 => 



\Q[p]-Q[p'}\ 

Qmax 



< e 



(1) 



Here Q max is the maximum of the functional. Lesche could 
show that, under this strict definition of stability, Renyi 
entropy is unstable. Indeed, he could find examples for p 
and p' for which — by taking the W — * oo limit — Lesche- 
stability is violated [1] . There it is also mentioned that the 
Li-norm has to be used because in this norm some relevant 
statistical quantities become independent of the number 
of states W. Taking the W — > oo limit is essential. If one 
can show that, in the W — » oo limit, stability is violated, 
this implies that for some finite W violation is already 
emerging, i.e. the bound e in Eq. (1) gets violated for 
specific distributions p and pf with | \p — p' \ \i < 5. Hence, 
the condition is not true for all W , the functional thus 
is Lesche-unstable. Lesche-stability has been used lately 
to analyze the stability of various generalized entropies. 
The use of entropies on that basis was suggested as a new 
validity criterion [1-4]. It is therefore not surprising that 
it has occasionally lead to some confusion, see e.g. [5,6]. 
The discussion of Lesche-stability has also been extended 
to other entropies [7,8]. If one does not divide by Q max , 
Eq. (1) becomes the traditional continuity condition for a 
functional and the stability criterion becomes a notion of 
uniform continuity. 
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The requirement that a functional Q[p] should only be 
considered stable if Eq. (1) holds for all p and p' and W 
uniformly, is unnecessarily strict for physical systems. In 
a physical context it is reasonable to call a functional Q[p] 
physically robust if it is continuous on the domain of phys- 
ically admissible distributions p. In the case of continuous 
distribution functions, admissible requires that the cor- 
responding Boltzmann Gibbs (BG) entropy (— J p\np) is 
positive 1 . In other words, for physical situations it is suf- 
ficient to ensure robustness of functionals, not necessarily 
their stability. 2 Although unbounded physical distribu- 
tions exist, we are not focusing on them here. For all other 
physically admissible distribution functions robustness is 
guaranteed. For finite W any probability distribution is 
admissible. 

The discussion of stability and robustness is not re- 
stricted to entropies [2-8], but also to other quantities 
such as the ^-expectation values, which naturally occur in 
the context of formalisms using generalized entropic func- 
tionals [9-12]. The g-expectation values (i.e., standard ex- 
pectation values with the so-called escort distribution [13], 
proportional to p q ) naturally appear in differential ther- 
modynamic relations whenever the probability distribu- 
tion presents power-law behavior. This can be illustrated 
with the ^-exponential function, e q (x) = [1 + (1 — q , )x]~ , 
which, for q > 1, asymptotically decays like a power-law. 
Indeed, whenever one takes derivatives of usual expecta- 
tion values escort expectation values can not be avoided, 
since the exponent q emerges due to de q (x)/dx — [e q (x)] g . 
For instance, normalization of the typical g-exponential 
distribution p(e) = e q (—a — /3e), where fi is the inverse 
temperature and a a normalization constant, requires 
1 = J °° dee q (—a — fie). A simple calculation shows that 
the derivative da/dfi = — J dep q (e)e/ J dep q (e), which is 
exactly the escort expectation of e. Another aspect can 
be illustrated with unimodal distributions. For example, 
if one has a g-Gaussian distribution [14], its width can be 
characterized by the (variance) 1 ^ 2 as long as q < 5/3. 
This is not true anymore if q > 5/3 (e.g. for q = 2, 
which corresponds to the celebrated Cauchy-Lorentz dis- 
tribution) since the variance diverges. In all cases, how- 
ever, we can characterize the width by the inverse of the 
maximal value of the distribution. It happens that this 
inverse scales like the (q — variance) 1 ^ 2 . 

Recently it was shown, using Lesche's two explicit exam- 
ples for p and p' [1], that ^-expectation values are unstable 
on discrete infinite distributions [15]. The first example 
corresponds to < q < 1, the second one to q > 1: 



1 In the continuum, entropy functionals such as the BG, Renyi 
and others are well known to become negative for distributions which 
include too narrow peaks, a situation which typically corresponds to 
the low temperature limit, where the quantum nature of physical 
systems must be taken into account. 

2 If a functional is stable it is always safe to use. Inversely, in- 
stability points at the fact that the domain of safe usage is limited. 
Robustness is never used in the sense of trajectories or attractors. 



Example (1): < q < 1 

/ (, t W \ SI 
Pi = Su > Pi={ 1 -2W^l) Pi+ 2W^l (2) 

Example (2): q > 1 

Pl = w^~T {1 ~ Stl) ' p ' i = { 1 ~l) Pi + l 5il ■ (3) 

Here \\p — p'\\i = 6, for any value of W. Specifically, 
in [15], instability was shown for the g-expectation of an 
observable O — {Oi}™^ 1 on the discrete index set / = 
{1, ...,W}, i.e., the expectation, with q ^ 1, of Q[p] = 
J2i P^Oi, where the escort distribution is given by 



For both examples limvi/^oo \Q[p] — Q[p']\ — \0 — 0\\, 
where O = limiy->oo W^ 1 J2i ®%i which proves instability 
when O and K are chosen such that \0 — 0\\ > K > 0, 
[15]. This implies that g-expectations are not uniformly 
continuous functionals in the limjy — > oo. It was con- 
cluded in [15] that the instability of the g-cxpectation 
value is the general situation, thus suggesting to re-think 
the use of g-expectation values in nonextensive statistical 
mechanics. While the result in [15] is correct in the strict 
sense of stability used there, this does not imply that q- 
expectation values are not robust either on finite sets - as 
will be shown here - or for continuous variables with the 
mentioned physical admissibility and boundedncss condi- 
tions [16]. Therefore the final conclusion drawn in [15] 
that the ^-expectation is in general unstable under small 
variations of the probability distributions docs not hold 
for physically relevant cases such as continuum distribu- 
tions and discrete distributions on finite support. Even 
for a discrete infinite support, robustness is verified, as 
it will be shown, for paradigmatic physical distributions. 
Robustness in the above sense is sufficient for virtually all 
practical physical purposes. In other words the require- 
ments of boundedncss and positive entropy exclude the 
pathological cases of singular distributions and singular 
variations. The examples used in [15] arc representatives 
of such pathological cases. 

In this contribution wc primarily discuss stability and 
robustness of ^-expectation values and Renyi entropy for 
three types of support for distribution functions, the con- 
tinuous, discrete finite, and discrete infinite. For the con- 
tinuous case, under the requirement of physically admissi- 
ble and bounded probability distributions and variations, 
we show the robustness of ^-expectation values and Renyi 
entropy. We then show that the theorems used in [16] al- 
low to prove robustness for all finite discrete sets. Even 
though it is not possible to immediately use the theorems 
to make statements about infinite discrete sets, which have 
been shown to be unstable for Renyi entropy and the q- 
expectation value (for q ^ 1) [1, 15], we show how the the- 
orems can be used to derive restrictions so that robustness 
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can be accomplished there as well. We finally discuss the 
situation for S q entropy for continuous and discrete finite 
probability distributions. A discussion on the distinction 
of discrete finite and infinite cases has been presented on 
numerical grounds in [17]. It is known that the nonaddi- 
tive entropy S q on discrete infinite distributions is robust 
because it is Lesche-stable [2]. 

Stability criteria for the g-expectation value 
for admissible continuous distribution functions. 

To make the paper self-contained we first review the 
stability criteria for the continuum case as discussed in 
two theorems in [16]. These two theorems determine the 
robustness criteria for ^-expectation values in the contin- 
uum. These theorems will be used below to show that not 
only the two examples of Lesche used in [15] are robust 
on finite sets, but that this is the case for all distribution 
functions on finite sets. 

For notation, in the continuum, the escort distri- 
bution reads P^ q \x) = j-g~^~^7j5 , where p denotes a 
continuous probability distribution. The expectation 
value of an observable 0(x) under this measure is 
Q[p] = J dxP( q \x)0(x), and its total variation reads 
SQ[p] — Q[p + 6p] — Q[p\- Here we use Q[p] to distinguish 
from the discrete case. 

The case < q < 1. The following theorem proves 
that, for < q < 1, instability only can happen for singu- 
lar distributions p. In the theorem ||C||oo = sup{|C?(a;)| : 
x G [0,1]} denotes the so called supremum or uniform 
norm, which is just the smallest upper bound of \0\. 

Theorem 1. Let < q < 1. Let < p be a non-singular 
probability distribution on L = [0, 1]. Let G = Jjdx p{x) q 
and let < 5 q = pG/A, for < p < 1, and Sp be a 
variation of the distribution such that, J T dx\5p\ = 5 < 5, 
and < p + 6p is positive on I. Furthermore, let < O 
be a strictly positive bounded observable on I, then there 
exists a constant < c < oo, such that 

\Q[p]-Q[p + 6p]\ <cS q . (5) 

Moreover, c = 4G- 2 ||0|| CO (1 + ||0||oo||0 _1 ||oo)/(l - p). 

The theorem states that, for positive bounded observ- 
ables, ^-expectation values are robust whenever the dis- 
tribution p is non-singular. 3 The class of singular dis- 
tributions is therefore the only class of distributions that 
contain all possible violations to stability for < q < 1, 
as long as the observable O is bounded on domain /. The 
corresponding example in [15] explicitly converges toward 
a singular distribution in the W —* oo continuum limit 
and thus violates stability. 

3 When all considered p are bounded by the same bound < p < 
B, then the constant c does not depend on the choice of p and Q is 
absolutely continuous on this domain. 



The case q > 1. In contrast to the < q < 1 case, 
instability for q > 1 is not primarily due to singular dis- 
tributions p, but due to the variation dp having singu- 
lar parts, i.e., due to an unbounded Sp. Note, that for 
bounded dp to exist, p also has to be non-singular. To 
keep J dx[p(x)) q and J dx[p(x)] q O finite, we further re- 
strict p to be bounded. 

Theorem 2. Let q > 1 and let m > be an arbitrary 
but fixed constant. Let < p be a probability distribution 
on L = [0, 1]. Let dp be variations of p, i.e. p + Sp > 0. 
Let B > be an arbitrary but fixed constant. Let the 
variations Sp be uniformly bounded in the m-norm, i.e. 
\\fip\\m < B, by this constant B. Further, let \\Sp\\\ = 5 
and let < O be a strictly positive bounded observable on 
L. Let 5 be an upper bound for the size of the variations 5 
such that, (2 1 /? - l^/T^-TllOHoollO-^loo)- 1 ^ > 5 > 

0, where 7 = (m — q)/(m — 1), then there exists a constant 
< R < 00, such that 

\Q[p]-Q[p + Sp}\<R6^ q , (6) 

and R does not depend on the choice of p. 

Theorem 2 states that, for positive bounded observ- 
ables, g-expectation values are robust whenever the distri- 
butions p are uniformly bounded 4 . Excluding unbounded 
variations from consideration therefore is sufficient to 
guarantee stability for the q > 1 case in a general set- 
ting. In the corresponding example in [15], when formu- 
lated in the continuum limit, stability is violated by using 
unbounded variations. 5 

Both theorems in [16] are analytical statements about 
the continuity properties of the ^-expectation value as a 
non-linear functional without any reference to thermody- 
namics. They provide a useful and flexible mathematical 
tool to analyze continuity properties for discrete sets as 
well as for the continuous case on [0, 00]. 

Stability criteria for the ^-expectation value for 
discrete finite probability functions. We can 

now use the above theorems to prove the robustness of 
q-cxpectations for bounded observables on finite discrete 
sets i E Lw = {1, 2, . . . , W}. For this we map the 
discrete probability distribution onto the continuous 
interval x G [0, 1] by identifying probabilities with 
probability densities p(x) = Wpi for x G [(i — 1)/W, i/W], 

1. e. step-functions on [0,1]. The observable {Oi}^ 1 
gets identified with the step function f(x) = Oi for 
the associated interval x G [(i — 1)/W,i/W]. Clearly, 
< p < W for all possible distributions of this kind and 

4 When all considered distributions are bounded by the same up- 
per bound (m — > 00 and 7 — ► 1) then R can be chosen independent 
of p and Q becomes uniformly continuous 

5 The proof was carried out on the unit interval / £ [0, 1]. This 
docs not present a loss of generality, since the proofs can be extended 
to any bounded interval. For unbounded intervals, the proof gets 
more involved and requires to fix conditions that relate boundedness 
conditions of the observable and the decay properties of p. 
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so are all possible variations since \p(x) — p'{x)\ < W. 
Since the observable f(x) is bounded, all conditions 
needed for theorems 1 and 2 are met. Let 7 = q for 
< q < 1 and 7 = 1/q for q > 1. For some given 
constant < 6 there exists a constant C, such that for 
all \p - p'\ < S < 5 it follows that C5~< > \Q[p] - Q[p'}\ 



where Q [p] 



^dx\p{x)] q f{x)/ ^dx[p{x)] q 



IS 



the ^-expectation for the continuous case. Now 

Q[p] = EZiW-^WptfOi/zZiW-^Wptf = 

Y^=\ViOilY^=\V\ = Q\p\, where Q[p] is the q- 
expectation on Iw Moreover, \\p\\i — J Q dx\p(x)\ = 

Y^=i \Pi\ — I bill an( i consequently one can pull back the 
result to the discrete case, which completes the proof. 

Comments on discrete infinite probability func- 
tions. — For discrete infinite distribution functions it 
was shown [15] that the ^-expectation value is unstable. 
However, it is possible to state conditions under which ro- 
bustness can be ensured. This can be done, for instance, 
in the following way. 

Discrete finite sets are closely related to continuous 
compact sets in the sense that discrete sequences can be 
mapped into the compact interval with step functions, as 
discussed above. In the same spirit, discrete infinite sets 
are intimately related to the continuous unbounded set 
[0,oo], since again one can map the discrete infinite se- 
quence into the continuous case in terms of step func- 
tions. If one can find conditions which define classes of 
distribution functions on [0, 00] that guarantee continu- 
ity or absolute continuity of the functional, i.e., the q- 
expectation, the same conditions are sufficient for prob- 
abilities on infinite discrete sets. Such classes can sim- 
ply be derived by using suitable differentiable monotonous 
functions, g : [0, 00] — > [0,1]. Let g' denote the deriva- 
tive of g and g^ 1 the inverse function of g. This maps 
the distribution function p, defined on [0, 00], to a dis- 
tribution function p(y) = p(g~ 1 (y))g'(g~ 1 (y))~ 1 on [0,1] 
and also the observable O on [0, 00] gets mapped to 
&{y) = 0(g- 1 (y))g'(g- 1 (y)) q - 1 . Now one can apply the 
conditions used for the theorems 1 and 2 on [0, 1] and pull 
them back to [0, 00] . It should be noted that different maps 
g lead to different, yet consistent boundedness conditions 
and decay properties for observables O and distributions 
p on [0,1]. 

To give an explicit example, let us consider the following 
problem. Suppose we consider g-exponential distributions 
of the form p{x) oc e,(— /3x) = [1 — (1 — q)/3x] ~ for q > 1 
and some [3 > (3q, and we want its first N moments under 
the g-expectation, 



Jdx[p(x)] q x n 
Jdx[p(x)]i 



(7) 



to be robust with respect to g, (i.e. m < N). For a more 
detailed discussion on moments under ^-expectations, see 
[18]. Assuming q > 1, let us take g(x) = 1 - 1/(1 + x) 7 ; 
then g'(x) = 7(1 + x) 1 . The boundedness condition 



for the observables immediately requires 7 > N/(q — l) — 1 
and the decay property for the distributions implies q < 
1 + 1/(7+1). For q = 1 and m — 1 the example is the q- 
expectation value of the energy of the quantum harmonic 
oscillator, 



E^[£l_T = i sinh - 2 (to 



(8) 



which shows continuity in (3 and robustness of the q- 
expectation under variations of the exponential distribu- 
tion. 6 

Robustness of Renyi entropy for continuous and 
discrete finite distribution functions. Here it suf- 
fices to discuss the case < q < 1 since for q > 1 Renyi 
entropy 



1 



(9) 



strictly speaking ceases to be a proper entropy because it 
is not concave. Substituting the probabilities Pi by step- 
functions p( w \x) = W P i < W for x e [(i - l)/W,i/W] 
which represent the discrete probability pi as a probability 
density on [0, 1], we get 



Sf = \nW + 



InfldxjpWjxW 



(10) 



Boundedness of the distributions allows to use propo- 
sitions (4), (6) and (8) in [16] which have been used to 
prove theorem (1). Note that, due to the upper bound W, 
it follows that / dxp{x) > W 1 - 1 . Let <5 = pW^/A and 
\\5p\\\ = 5 < 5, as in [16]. Now we get 



\Sf[p]-Sf[p + Sp]\ 



In 



Sldx[p{x)Y 



Jo dx [p(x) + 5p(x)]i 



■ (11) 



Using propositions (4) and (6) from [16] one finds 1 — 
a5i < I J*^ dx p{x) q / dx (p(x) + 5p{x)Y\ < 1 + aS q and 
a = AW 1 - q /{\ - p). Since | ln(l + x)| < 2|x|, for |x| < 1 
it follows that for sufficiently small 5, 



\Sf[p}-Sf[p + Sp]\<2a5 q 



(12) 



This shows both the uniform continuity of the continuous 
Renyi entropy for the class of uniformly bounded probabil- 
ity distributions in Li([0, 1]), and the absolute continuity 
of Renyi entropy for probabilities on finite sets. Similar 
arguments show robustness also for q > 1, despite lack of 
concavity of . 



6 In what concerns the use of continuous distributions for calcu- 
lating entropies and similar quantities, such as Eq. (7), the reader 
must be aware of a relatively well known difficulty. If we make a 
change of variables y = f(x), say in Eq. (7), we immediately see 
that the result is not invariant. In the spirit of Kullback and Leibler 
entropy [19] the problem is easily resolved in terms of a reference 
distribution r(x), which, except at infinity, nowhere vanishes. For 
example, for q > 0, the quantity in Eq. (7) would be replaced by 

f dxr(x)[p(x)/r(x)] q x m 
J dxr(x)[p(x) /r(x)]1 
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Robustness of the entropy S q for continuous and 
discrete finite distribution functions. One can 

prove robustness of the nonadditive entropy 



T w v q 



q-1 



(13) 



for discrete finite sets by mapping the probabilities {pi}^ 
onto a distribution on p on [0, 1] by step functions, as 
above. Since all step functions p that represent some 
feli^i are bounded by W, it is sufficient to prove ro- 
bustness for the continuous case, p is from a uniformly 
bounded class of distribution functions on [0, 1], i.e., there 
is a constant W > such that all p are bounded by W . 
Consider variations Sp = p' — p such that both p and p' 
are bounded by W and \\Sp\\i = S, where S is sufficiently 
small. Now, \S q [p] - S q [p + Sp}\ = \q - l^l £ dx p" - 
Jq 1 dx (p + Sp) q \ . It therefore is an immediate consequence 
of propositions (6) and (4) in [16], that for < q < 1, 
\S q [p] — S q [p + Sp]\ < 45 q . For q > 1 we use propositions 
(11) and (13) to find \S q [p\ - S q [p + Sp}\ < R5«. 

In what concerns discrete infinite distribution functions, 
S q has been shown to be Lesche-stable [2] . 

Conclusion. — To summarize, we discussed the con- 
cept of physical robustness in contrast to the more re- 
strictive mathematical stability of thermostatistical func- 
tionals under variations of their underlying distribution 
functions. We argue that while several important func- 
tionals, such as the Renyi entropy or the ^-expectation 
value are unstable in the strict sense, restriction to phys- 
ically relevant distribution functions, ensures robustness 
of these functionals. For a distribution to be physically 
relevant we require that its associated entropy be non- 
negative. We further restrict to distributions which are 
bounded in the continuum. This excludes, for example, 
distributions involving Dirac deltas. Our results are sum- 
marized in Table 1, where we indicate the type of stability, 
Lesche-stability or robustness for the functionals S q , Renyi 
entropy and the g-expectation value, for the paradig- 
matic types of distribution functions - continuum, discrete 
finite, and discrete infinite - that we have focused on here. 
The term Lesche-stable is used when Eq. (1) holds, stable 
refers to the situation where no division by a maximum is 
taken in Eq. (1), and robustness - in the above-defined 
sense - is found for admissible (non-negative entropy) and 
bounded distributions in the continuum and for discrete 
finite probability functions. In the case for discrete infinite 
distribution functions which are known to cause instabil- 
ities for some functionals [1,15], one can show that there 
exist paradigmatic robust examples once decay properties 
of the distributions are specified. In this context one can 
show that systems such as the harmonic oscillator are ro- 
bust under variations of the (inverse) temperature. One 
can of course think of physical distribution functions which 
have positive BG entropy but are unbounded, such as e.g. 
power-law divergences. These cases remain to be discussed 



but exceed the present scope. 

We conclude by stating that the concept of stability 
might be overly strict for physical applications. This is in 
accordance to conclusions drawn in [20] . When limited to 
the class of physically admissible and bounded distribu- 
tion functions, it is conceivable that physical robustness 
of virtually all thermodynamic functionals will be guaran- 
teed. 
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S q , for q > 


, for q > 


g-expectation value, for q > 


bounded continuous 
(physically admissible) 


robust 


robust 


robust [16] 


discrete finite 
(W finite) 


robust 


robust 


robust 


discrete infinite 
(lim W — > exo) 


Lesche-stable [2] 


Lcschc-unstablc [1]; robust 
for typical physical cases 


unstable [15]; robust for 
typical physical cases 



Table 1: Table of stability/Lesche-stability and robustness for the functionals S q , S q and the g-expectation value for the 
underlying nature of the distribution function, i.e. continuous, discrete finite or discrete infinite. In the continuum, admissible 
means that a non-negative entropy is required. Boundedness automatically guarantees robustness [16]. The term Lesche-stable 
is used for infinite distribution functions when Eq. (1) holds, stable is used when there is no division by a maximum taken in 
Eq. (1). Robustness is used for bounded distributions in the continuum and for discrete finite probability functions. For the 
g-expectations for the discrete infinite case, robustness is understood under the decay properties of the distributions, as specified 
in the text. Lesche-stability and stability are sufficient but not necessary for robustness, Sbg and standard expectation values 
(q = 1) are Lesche-stable and stable, respectively. 
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